The aim of auto-focusing is to position a camera lens so that a scene of interest is properly focused. Consequently, auto-focusing can be represented as an optimization problem including determining a focus measurement at a given lens position and moving the camera or the camera lens to improve the focus measurement. These two steps are typically integrated into a hill-climbing process and repeated until the focus measurement has been maximized.
The present invention is concerned with the first of the above steps, namely obtaining a focus measurement for a given lens position relative to a scene of interest. Accordingly, it is useful at this point to briefly review existing methods of focus measurement and the problems associated therewith, by referring to the accompanying Figures in which: FIG. 1 is a graph of the pixel intensity response profile of a pixel array to an edge between a bright region and a dark region in a scene, wherein the graph also shows the edge strength and contrast parameters of the imaged scene; FIG. 2(a) is a graph of the intensity (I) measured from two pixels (P1 and P2) as a function of exposure time (T); FIG. 2(b) is a graph of the intensity (I) measured from two pixels (P1 and P2) as a function of gain (G); FIG. 3(a) is diagram of an edge between two regions R1, R2 in an image obtained under bright and dark lighting conditions; FIG. 3(b) are graphs of the intensity profile of a pixel array to regions R1 and R2 in the images shown in FIG. 3(a); and FIG. 4 is an image of a cylindrical object against a background and sections taken thereof.
Review of Existing Methods of Focus Measurement
The process of focus measurement attempts to determine the degree of camera focus from an analysis of image data. A well-focused image is one in which edges are sharp (i.e. not diffuse) and the overall contrast in the image is maximized. These two observations lead to two methods of determining whether an image is in focus, namely edge-based methods and contrast-based methods.
Referring to FIG. 1, the pixel intensity I profile derived from the response of a pixel array to an edge between a dark region and a bright region is typically sigmoidal in shape. In particular, the pixel intensity I is greatest when imaging a bright region and smallest when imaging a dark region. The strength of the edge E between a bright and dark region can be described by the slope of the rising portion of the sigmoidal pixel intensity profile. Accordingly, when an edge in an image is very sharply defined, the rising portion of the pixel intensity profile is very steep, whereas if the edge is diffuse or less sharply defined, the rising portion of the pixel intensity profile is correspondingly shallow. The contrast C in the imaged scene can be described as the difference between the maximum and minimum intensity signals from the pixel array.
Since the measurement of edge strength is restricted to the rising portion of the pixel intensity profile, whereas contrast covers the entire pixel intensity profile, the number of pixels used for calculating edge strength PE is typically less than the number of pixels used for calculating contrast PC.
Edge-based methods of focus measurement are based on the premise that an edge detector is more likely to detect the edges in a properly focused image than in an unfocused image. Or in other words, the output of an edge detector when applied to an image should be greater when the image is properly focused than when poorly focused. Contrast-based methods of focus measurement are based on the premise that the overall contrast in an image is maximized when the image is properly focused.
Limitations of Existing Methods of Focus Measurement and Practical Effects Thereof
Ideally, a focus measurement should be independent of the illumination or content of a scene. However, both edge-based and contrast-based methods of focus measurement provide relative focus measurements obtained from comparisons between frames. Accordingly, the value of the focus measurements produced by the edge-based and contrast-based methods vary if the exposure time of the camera or the scene illumination or content is altered. Consequently, the above-mentioned auto-focusing optimization procedure must be halted or re-initiated if the content or illumination of a scene changes.
Furthermore, since relative focus measures vary in accordance with the content of a scene, separate regions of a scene cannot be directly compared. This may be particularly problematic if a scene contains several objects positioned at different distances relative to the camera. In addition, since edge-based and contrast-based methods of focus measurement provide inherently relative measures of focus, they generally do not provide information regarding the absolute focus quality of a given image, but instead provide information indicating whether focus has improved or deteriorated between frames.
The above-mentioned limitations of conventional edge-based and contrast-based methods of focus measurement will be discussed in more detail below.
Edge-Based Methods of Focus Measurement
In its simplest implementation, edge-strength can be determined by measuring differences in intensity between adjacent pixels. However, this measurement is affected by exposure time and gain changes in a light sensor as will be discussed below.
Effects of Exposure Time: Referring to FIG. 2(a) while the intensity I measured from pixels P1 and P2 increases linearly with the exposure time T of the corresponding light sensors, the nature of the linear relationship is not the same for each pixel. In particular, the slope of the intensity-exposure time graph for pixel P2 is much larger than for pixel P1. Consequently, while a given increase ΔT in exposure time increases the intensity of pixel P1 by a delta of ΔP1, it increases the intensity of pixel P2 by a delta of ΔP2 wherein ΔP2>ΔP1.
Effects of Gain: Referring to FIG. 2(b) at gain G1 the intensity measured from pixel P2 is larger than that measured from pixel P1. Similarly, on increasing the gain to G2, the intensity measured from pixel P2 increases by a greater amount than that of pixel P1 (i.e. the intensity of P2 increases by a delta of ΔP2 whereas the intensity of P1 increases by a delta of ΔP1, where ΔP1<ΔP2). In other words, the variation of pixel intensity with gain differs according to the absolute value of the intensity of a given pixel. Consequently, the application of a gain to a number of pixels increases the intensity difference between the pixels by that gain factor.
With no guarantee that similar gains will be applied from one test to the next, different intensity measurements (and thus focus measures) can be obtained for the same scene.
Contrast-Based Methods of Focus Measurement
FIG. 3(a) shows an idealized image obtained by a properly focused camera from a scene under bright and dark lighting conditions. When the scene is brightly lit (case A), there is a clear and definite contrast between regions R1 and R2 in the acquired image. However, when the scene is less brightly lit (case B) the contrast between regions R1 and R2 in the acquired image is not as clearly defined.
Referring to FIG. 3(b) the intensity of the pixels that acquired the image in regions R1 and R2 (in FIG. 3(a)) possesses a sigmoidal profile. However, the sigmoidal pixel intensity profile in case A is much steeper than in case B. In other words, the slope mA of the rising portion of the pixel intensity profile in case A is considerably larger than in case B (i.e. mA>mB). Accordingly, since contrast-based methods of focus measurement directly relate focus measurements to contrast, the focus measurement obtained in the case A is larger than that obtained in case B despite the fact that both cases are equally well focused.
Practical Effects of Limitations of Existing Methods of Focus Measurement
The above-described limitations of edge-based and contrast-based methods of focus measurement are not merely theoretical abstractions. These limitations have practical effects on the focus measurements acquired from scenes as will be described below.
Effects of Shape of Imaged Object on Edge Based Methods of Focus Measurement
In a scene comprised of multiple objects disposed at the same distance from a camera, each edge of each object should possess the same objective focus measurement. However, in practice the shape characteristics or reflectance of an imaged object can affect the edge-strength (and thus focus measurement) detected therefrom.
Referring to FIG. 4, an image is acquired of a cylindrical object 10 against a background 11. For the purpose of the present example, it will be assumed that the object 10 is equally focused over its entire area. A first section 12 of the image depicts the interface between the object 10 and the background 11. A second section 14 of the image depicts a portion of the curved surface of the cylindrical object 10. Since the first section 12 possesses elements of the object 10 and the background 11, it typically has a larger edge strength value than the second section 14, because the second section 14 does not contain enough differential and contextual information to enable a strong edge to be detected.
As previously discussed, edge-based methods of focus measurement obtain a focus measurement by consideration of average edge values in a scene. However, as shown above, an image can possess widely varying edge values depending on the shape of the imaged object. Accordingly, the average edge-value obtained from a single image may contain contributions from the multiple edge values in the image. Consequently, the average edge-value obtained by the edge-based method of focus measurement blurs and loses the information from the multiple edge value measurements in an imaged scene.
Effect of Multiple Objects Positioned at Different Distances From a Camera
Where a region-of interest (or a whole scene) contains objects at different distances from a camera, the focus of each of these objects should be different. However, the average focusing effect of conventional edge-based and contrast-based focus measurement techniques generates an intermediate focus measurement that is likely to be incorrect for any and/or all of the objects in the scene.